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INTRODUCTION 


Neural  networks  have  undergone  a  resurgence  of  late,  due  to  the  work  of 
Hopfield,  Grossberg  and  others.  They  have  the  potential  for  applications  in  a 
wide  range  of  areas,  including  signal  processing,  image  processing,  planning, 
control,  and  data  fusion.  In  this  paper  we  wiU  address  the  applications  being 
explored  at  NOSC.  We  will  also  describe  our  experiences  with  two  neural 
network  products  currently  being  marketed:  the  Hecht-Nielsen  Anza 
Neurocomputer  and  the  SAIC  Sigma-1  Neurocomputer. 

The  projects  will  be  grouped  by  the  NOSC  branch.  In  each  case  a  point  of 
contact  will  be  given,  though  of  course  there  may  be  others  working  on  each 
project.  From  the  wide  range  of  problems  being  addressed,  it  is  clear  that  the 
Navy  has  a  strong  interest  in  neural  networks,  and  believes  that  they  are  well 
suited  to  a  wide  range  of  applications.  The  authors  are  in  Code  421,  and  as  such 
have  complete  knowledge  of  the  neural  network  research  ongoing  in  Code  421. 
The  descriptions  of  the  work  being  done  elsewhere  at  NOSC  are  based  on  varying 
degrees  of  interaction.  For  many,  first  hand  knowledge  of  the  research  is  present. 
The  authors  are  in  fact  working  directly  with  many  groups  outside  their  own 
code.  For  others,  the  interaction  is  not  as  thorough.  In  any  case,  the  interested 
reader  should  contact  the  specific  group  indicated  for  a  more  in  depth  discussion 
of  the  work  described  herein. 

Command  and  Control  Department 

Information  Systems  Division 

Code  421:  Architecture  and  Applied  Research  Branch 

Data  fusion  is  of  great  interest  to  the  Navy,  although  the  description  of  data 
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fusion  is  quite  nefarious.  There  are  two  aspects  of  data  fusion  being  explored  via 
neural  networks  in  code  421:  Hull-to-emitter  correlation  (hulltec)  and  situation 
assessment. 


Hulltec  involves  determining  the  origin  of  a  signal  from  parameters  measured 
from  the  signal.  Neural  networks  have  been  developed  which  compare  favorably 
with  traditional  methods  of  hulltec.  A  backpropagation  network  was  used 
initially,  but  a  new  architecture  has  been  developed  which  allows  the  network  to 
learn  the  statistical  distribution  of  the  signals. 


The  problem  of  situation  assessment  is  quite  different.  Here  the  system  is 
given  reports  from  several  sources  and  must  determine  the  events  of  interest  that 
are  taking  place  in  the  area.  An  unclassified  version  of  this  is  being  explored  in 
which  a  sentry  robot  incorporating  several  types  of  sensors  on  a  mobile  platform 
is  required  to  determine  when  an  intruder  has  entered  its  area.  This  involves  the 
fusion  of  data  from  several  different  types  of  sensors  as  well  as  different  sensors 
within  each  type.  Both  these  problems  require  the  system  to  be  able  to  store  and 
recall  patterns  and  sequences  and  to  do  recognition  based  not  on  static  data  but  on 
time  varying  data.  A  network  with  this  capability  has  been  developed. 


The  above  network  is  being  tested  on  a  speech  recognition  task:  isolated  word 
recognition.  The  system  must  recognize  the  word  being  spoken  from  the 
waveform.  The  eventual  goal  is  to  expand  this  research  to  spe^er  independent 
word  recognition  and  continuous  speech  recognition. 


A  network  to  do  translation  invariant  pattern  recognition  is  also  being 
developed.  The  goal  is  a  system  that  can  recognize  a  hand  printed  character  in  any 
position  within  the  field  of  view,  and  to  recognize  multiple  characters  within  the 
field  of  view.  Two  approaches  are  being  explored:  a  simulation  of  Fukushima's 
Neocognitron  is  being  developed,  and  a  similar  system  using  gaussian  feature 
detectors  is  also  being  explored.  An  adaptation  of  this  system  will  be  employed  in 
the  robot  described  above  to  allow  attention  focussing. 


Advanced  Technologies  Division 

Code  441 :  Human  Factors  and  Speech  Technology  Branch 

Steve  Luse  and  his  co workers  have  two  neural  network  projects  in  the  area  of  r  ~ 
speech  processing.  The  first  involves  determining  the  optimum  configuration  of  a 
backprop  network  as  a  preprocessor  for  speech.  The  network  parameters  that  are  □ 
being  investigated  are  number  and  size  of  hidden  layers.  The  second  project  ,  ^ 

involves  using  Hecht-Nielsen’s  spatiotemporal  network  to  recognize  isolated  ~ 

spoken  digits.  This  project,  like  their  other  network  project,  is  involved  with _ 

determining  the  utility  of  the  given  architecture  for  the  problem,  rather  than  /  _ 

constructing  an  architecture  to  solve  the  problem.  As  could  be  expected,  the  y  codes 
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network's  performance  on  test  digits  ranged  from  mediocre  to  very  good. 

Code  444:  Artificial  Intelligence  Technology  Branch 

A  project  similar  to  our  character  recognition  problem  is  being  undertaken  by 
Chris  Haupt.  Here,  the  problem  is  to  categorize  images  from  Inverse  Synthetic 
Aperture  Radar.  Chris  is  looking  at  several  approaches,  ranging  from  traditional 
correlation  techniques  to  a  variation  of  code  421's  temporal  processing  system.  In 
addition  to  the  image  problem,  he  is  working  on  an  implementation  of  Hopfield's 
temporal  sequence  recognizer,  and  this  will  be  used  to  compare  and  contrast  with 
code  42rs  temporal  system. 

Marine  Sciences  and  Technology  Department 

Solid  State  Electronics  Division 

Code  551:  Design  and  Develonment  Branch 


In  an  attempt  to  address  both  the  problem  of  slow  simulations  and  the  need  for 
delivery  systems,  Pat  Shoemaker  and  his  coworkers  are  designing  and  fabricating 
a  neural  network  chip  which  will  implement  the  weight  changes  in  silicon.  The 
architecture  chosen  for  implementation  is  the  backpropagation  model.  The 
weights  will  be  stored  in  variable  resistors  and  the  system  will  allow  modification 
of  these  weights  using  the  generalized  delta  backpropagation  rule.  After  learning, 
the  weights  can  be  "frozen"  to  allow  the  chip  to  be  fielded  in  an  application  where 
learning  is  not  appropriate  after  the  initial  training.  When  the  environment  in 
which  the  chip  is  operating  changes,  the  chip  can  be  retrained  to  handle  the  new 
information. 

ASW  Department 

ASW  Technology  Division 
Code  633:  Research  Branch 

Chaos  is  a  hot  topic  in  physics,  and  Adi  Bulsara  and  his  colleagues  have  taken 
their  expertise  in  this  field  and  are  working  on  applying  it  to  neural  nets.  The 
project  intends  to  look  at  the  mathematical  models  of  biological  neurons  and  study 
the  conditions  under  which  chaos  will  occure  in  the  system.  Future  work  will 
include  the  study  of  small  networks  of  these  neurons.  Different  models  of  the 
neuron  will  be  explored  and  simulated,  and  an  analytical  description  of  the  onset 
of  chaos  wUl  be  attempted. 

ASW  Systems  Engineering  Division 


Steve  Speidel  has  worked  on  a  neural  network  architecture  to  do  adaptive 
beamforming.  He  is  currently  proposing  a  project  to  use  a  network  to  interpret 
sequences  of  multipath  sound  reflections  in  terms  of  reflector  position. 

ASW  Analysis  and  Simulation  Division 
Code  663:  Modeling  and  Analysis  Branch 

Bob  Bologna  and  his  colleagues  have  investigated  the  utility  of  both  the 
backpropagation  and  the  adaptive  resonance  paradigms  for  acoustic  signal 
processing.  They  currently  have  no  funded  project  using  neural  nets,  but  are 
active  in  teaching  the  technology  to  new  employees  under  the  New  Professional 
program. 


Surveillance  Department 

Signal  and  Information  Processing  Division 

Code  731:  Information  Processing  and  Display  Branch 

Curt  Goodhart  and  his  coworkers  have  been  studying  the  backpropagation 
network.  They  have  been  applying  it  to  the  problem  of  identifying  transients  in  an 
acoustical  signal. 

Engineering  and  Computer  Sciences  Department 

Ocean  Engineering  Division 

Code  943:  Undersea  AI  and  Robotics  Branch 

This  branch  is  working  with  an  underwater  teleoperated  vehicle  testbed, 
which  allows  researchers  to  add  functionallity  such  as  sensors,  processors  and 
software  to  increase  its  autonomy.  Neural  networks  are  being  applied  to  the 
vision  system.  Using  a  model  of  the  early  visual  system,  Mike  Blackburn  and  his 
associates  are  building  a  visual  processor  for  motion  detection.  The  rational  for 
this  is  that  in  the  ocean,  things  Aat  move  tend  to  be  interesting,  and  so  a  system 
that  can  detect  and  identify  moving  objects  is  of  value  for  a  free  swimming  or 
tethered  vehicle. 


Neurocomputers 

One  of  the  drawbacks  of  neural  network  research  is  the  processing 
requirements  of  the  simulations.  Often,  networks  must  be  applied  to  "toy" 
problems  because  the  time  required  in  simulation  is  too  great  to  deal  with  the 
scaled  up  problem.  This  can  be  a  serious  drawback,  as  it  is  often  nontrivial  to 
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scale  a  solution  up  from  a  "toy"  problem.  One  of  the  ways  that  industry  is 
addressing  this  problem  is  to  produce  add-on  boards  for  existing  systems,  so 
called  "neurocomputers",  which  give  the  new  system  greater  processing  power. 
We  have  had  experience  with  two  systems  which  provide  a  coprocessor  board  and 
network  software  packages.  Both  systems  will  also  offer  languages  to  better 
access  their  hardware,  but  these  languages  are  not  yet  available. 

Our  experience  with  "netware"  packages  is  that  they  are  of  limited  use  to  a 
researcher.  The  limitations  imposed  by  the  packages  we  have  used  are  such  that 
they  are  quickly  abandoned  for  more  flexible  C  programs.  In  our  experience,  no 
one  network  architecture  is  suitable  for  most  problems.  In  fact,  one  often  has  to 
piece  together  ideas  from  several  different  architectures,  or  invent  a  whole  new 
paradigm  for  each  problem.  The  so  called  "netware"  packages  we  are  familiar 
with  do  not  do  this. 

The  Hecht-Nielsen  ANZA  Neurocomputer  comes  with  a  netware  package  of 
several  popular  networks.  The  networks  are  packaged  as  a  C  library  of  functions  , 
and  some  demo  programs.  The  functions  allow  the  programmer  to  create  nets, 
initialize  weights,  tune  a  few  parameters  of  the  paradigm  to  fit  the  problem,  and 
run  the  network.  Since  these  functions  are  called  from  a  C  program,  any 
preprocessing  can  be  done  within  the  C  program.  However,  this  preprocessing 
must  be  done  on  the  host  machine,  in  our  case  a  PC-AT  clone  without  a  math 
coprocessor  (the  system  provided  by  HNC).  The  demo  programs  have  graphic 
displays,  but  these  graphics  functions  are  not  a  part  of  the  netware  package  and 
must  be  purchased  separately. 

The  networks  within  the  ANZA  package  are  too  limited  to  be  of  much  use,  in 
our  opinion.  For  example,  the  backpropagation  network  is  limited  to  a  single 
hidden  layer,  while  it  can  be  shown  that  some  problems  require  two  hidden  layers. 
Also,  the  activation  function  for  the  backprop  net  is  constrained  to  be  linear  or 
sigmoid,  though  there  are  reasons  to  choose  other  activation  functions  for  some 
problems. 

The  SAIC  Sigma-1  Neurocomputer  comes  with  a  netware  package  (ANSim) 
of  twice  the  number  of  networks  as  the  ANZA.  The  packages  run  within 
Microsoft  Windows,  and  the  user  is  able  to  tune  the  parameters  of  the  networks 
using  mouse  and  keyboard.  Over  all,  the  networks  in  this  package  are  more 
flexible  than  the  HNC  system,  but  they  are  more  cumbersome  to  use  if  pre-  or 
post-processing  is  desired.  The  ANSim  system  also  comes  with  graphic  displays 
that  allow  the  user  to  view  weights,  errors,  etc  on  the  screen.  This  is  useful,  but 
since  the  source  is  unavailable,  the  user  must  use  the  predefined  graphics,  which  is 
not  always  what  is  needed. 

Overall,  we  find  systems  of  this  type  to  be  of  limited  use.  The  ANSim 
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software  would  be  best  for  a  first  time  user,  particularly  someone  with  little  or  no 
programming  experience.  The  HNC  netware  is  a  step  in  the  right  direction  for 
researchers,  but  the  networks  are  too  limited  to  be  of  much  use. 

The  only  projects  at  NOSC  that  could  use  these  software  systems  are  the  ones 
that  chose  to  investigate  a  particular  network  paradigm.  Here  the  purpose  is  more 
to  become  familiar  with  the  technology  and  its  limitations  dian  to  solve  a 
particular  problem.  From  our  experience,  any  real  world  problem  requires  a 
hybrid  system,  rather  than  a  particular  network  paradigm. 

On  the  hardware  side,  not  much  can  be  said  at  this  time.  The  ANZA  is  a 
68020  coprocessor  board  which  cannot  be  accessed  except  through  the  netware 
package.  Again,  a  full  featured  language  has  been  promised,  but  a  simple  C 
compiler  would  in  our  opinion  be  a  minimum  requirement  to  make  the  hardware 
accessable  to  the  researcher.  A  C  compiler  is  available  from  the  company  that 
makes  the  ANZA  board,  but  it  did  not  arrive  in  time  to  be  included  in  this  review. 
The  SAIC  board  has  had  some  delays  and  we  have  been  told  that  it  will  be  ready 
for  release  in  the  next  few  weeks.  The  board  will  come  with  a  C  compiler  and  has 
(according  to  SAIC)  a  20  Mflop  processor.  The  SAIC  system  comes  packaged  in 
a  Wyse  80386  machine,  and  without  the  board  it  performs  at  close  to  the  speed  of 
the  ANZA  board.  The  ANZA  can  also  be  purchased  with  a  386  system. 

Conclusions 

Neural  networks  are  being  applied  to  a  wide  range  of  problems  here  at  NOSC. 
They  have  been  found  to  be  a  flexible  and  useful  tool  for  many  types  of  signal 
processing.  It  is  our  experience  that  most  of  the  architectures  available  can  do  a 
fair  job  on  most  problems,  but  that  a  network  architecture  can  often  be  designed 
to  fit  the  problem,  and  that  these  networks  can  do  as  well  or  better  than  traditional 
methods.  The  technology  is  young  and  very  promising.  The  inherent  parallelism 
and  fault  tolerance  make  it  a  very  attractive  tool  for  many  applications. 

Often,  the  biggest  problem  with  a  neural  network  solution  to  the  problem  is 
the  processing  time  required  of  simulations.  Industry  is  starting  to  produce  some 
impressive  coprocessor  boards  that  may  help  with  the  simulation  processing  load. 
In  our  experience,  the  important  factor  in  these  products  is  the  speed  and  memory 
capacity  of  the  hardware,  and  the  languages  available  for  the  board,  since  the  field 
is  too  young  for  the  software  packages  to  be  of  much  use.  NOSC  and  others  are 
producing  chips  which  will  take  advantage  of  the  parallelism  of  the  system  and 
allow  real  time  processing. 
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